Databricks Architect (AWS | Spark | PySpark)

ICONMA, LLC • United States, United States • 2h ago

Our client, a IT Servies and Consulting company, is looking for a Databricks Architect (AWS | Spark | PySpark) for their Bridgewater, NJ location. Responsibilities:

This Architect role focuses on designing and governing scalable data platforms using Databricks Spark PySpark and AWS Cloud for a global organization in a hybrid work model.

The Architect will own end to end data solution architecture oversee Databricks administration and optimize cloud resources to enable reliable analytics and machine learning outcomes that support critical business decisions and create societal value through data driven innovation.

Architect robust and scalable Databricks based data platforms that enable secure high performance data processing and analytics for business critical workloads in a hybrid work environment.

Design end to end data solutions leveraging Spark and PySpark that integrate batch and streaming data pipelines to deliver reliable curated data sets for downstream consumption across diverse teams.

Oversee Databricks workspace configuration including cluster policies access controls and environment standards to ensure consistent governance compliance and operational efficiency. Implement AWS Cloud architectures for data and analytics including storage compute and networking components to optimize cost performance reliability and alignment with enterprise standards.

Administer Databricks environments by monitoring job performance capacity utilization and error trends to proactively tune clusters and improve overall platform stability and throughput. Provide technical guidance on PySpark coding standards reusable frameworks and data transformation patterns to enhance maintainability reduce defects and accelerate delivery of data products.

Coordinate with data engineers data scientists and product teams to translate business requirements into data architecture blueprints that maximize value creation and support strategic initiatives.

Establish and enforce best practices for data security privacy and access management across Databricks and AWS including role based controls encryption and audit monitoring.

Evaluate and introduce optimization techniques for Spark and PySpark such as efficient partitioning caching and data format choices to reduce processing time and resource consumption. Collaborate with cloud operations teams to define backup recovery and high availability strategies for Databricks and AWS data components ensuring resilience and business continuity. Drive continuous improvement of data platform capabilities by assessing new Databricks and AWS features piloting enhancements and advocating adoption that benefits organizational goals and societal impact.

Create clear technical documentation including solution designs runbooks and standards so delivery and operations teams can implement and support data platforms with confidence and consistency.

Mentor peers and junior practitioners by sharing domain expertise on Databricks Spark PySpark and AWS Cloud helping build internal capability and fostering a culture of quality and innovation.

Requirements:

10+ years in Data Engineering, Data Architecture, or Cloud Data Platforms

5+ years of hands-on Databricks architecture and administration

Strong experience designing enterprise-scale data platforms on AWS

Pharmaceutical, Biotechnology, Healthcare, or other highly regulated enterprise environments

Experience supporting enterprise analytics, AI/ML, and large-scale data modernization initiatives

Experience: 12to16Yrs

Demonstrate extensive experience in Databricks administration including workspace setup cluster management governance configuration and troubleshooting in complex enterprise @contexts. Bring strong practical proficiency in Spark and PySpark for building scalable data pipelines transformations and performance tuned jobs applied to large and varied data sets.

Possess deep hands on experience with AWS Cloud services relevant to data platforms such as data storage compute analytics and security features to design integrated solutions.

Show capability in AWS administration including identity and access configuration monitoring resource optimization and adherence to organizational security and compliance policies. Apply solid understanding of data engineering principles data modeling and distributed computing concepts to craft architectures that are robust maintainable and future ready.

Utilize proven problem solving and analytical skills to diagnose platform issues optimize workloads and propose architectural improvements that enhance business value and user trust. Exhibit effective communication and stakeholder engagement skills to collaborate with cross functional teams explain architectural choices and align solutions with strategic objectives.

Preferred certifications include Databricks Certified Data Engineer AWS Certified Solutions Architect Professional or equivalent cloud and data engineering credentials.

Why Should You Apply?

Health Benefits

Referral Program

Excellent growth and advancement opportunities

Similar Jobs

Staff Product Data Scientist, Pixel Growth, Google Store

Google • Mountain View, CA • 2h ago

Retail & Hospitality

Staff Product Data Scientist, Pixel Growth, Google Store corporatefare_ Google place Mountain View, CA, USA Advanced Experience owning outcomes and decision making, solving ambiguous problems and influencing stakeholders...

2h ago

Go to Apply

Restaurant Leader (Manager) Trainee

7-Eleven • Hobbs, NM • 2h ago

Retail & Hospitality

Restaurant Leader (Manager) Trainee Apply Now ( https://my7elevenhr.wd12.myworkdayjobs.com/Careers/job/Store-42590-HOBBS-NM/Restaurant-Leader--Manager--Trainee\R26\4602/apply ) Save Job Job ID R26_4602 Store-ID 42590 Add...

2h ago

Go to Apply

Restaurant Host - Ruscello - Ridgedale

Nordstrom • Minnetonka, MN • 2h ago

Retail & Hospitality

Job Description The ideal host/hostess is self-motivated, friendly and committed to providing outstanding customer service every day. A day in the life... Greet and seat customers quickly and in a professional manner Sha...

2h ago

Go to Apply

Growth Assistant Restaurant Leader

Raising Cane's • Fort Lauderdale, FL • 2h ago

Retail & Hospitality

Company Description Starting from $55,000 annually plus monthly training incentive of $750 Pay is based on location, experience, and qualifications etc. *Monthly incentives after training vary and are based on restaurant...

2h ago

Go to Apply

Restaurant Crewmember - Closing Shifts

Raising Cane's • Omaha, NE • 2h ago

Retail & Hospitality

Starting hiring pay: $15.00 Restaurant Crewmembers at Raising Cane's will wear many hats (including a Raising Cane's hat) while working hard and having fun as a critical part of the Restaurant team. We are hiring immedia...

2h ago

Go to Apply

Restaurant Crewmember - Part Time

Raising Cane's • Pickerington, OH • 2h ago

Retail & Hospitality

Starting hiring pay: $14.00 Restaurant Crewmembers at Raising Cane's will wear many hats (including a Raising Cane's hat) while working hard and having fun as a critical part of the Restaurant team. We are hiring immedia...

2h ago

Go to Apply

Restaurant Crewmember - Nights/Weekends

Raising Cane's • Grove City, OH • 2h ago

Retail & Hospitality

2h ago

Go to Apply

Restaurant Leader (Manager) Trainee

7-Eleven • Comanche, TX • 2h ago

Retail & Hospitality

Restaurant Leader (Manager) Trainee Apply Now ( https://my7elevenhr.wd12.myworkdayjobs.com/Careers/job/Store-42521-COMANCHE-TX/Restaurant-Leader--Manager--Trainee\R26\4603/apply ) Save Job Job ID R26_4603 Store-ID 42521...

2h ago

Go to Apply

Growth Assistant Restaurant Leader

Raising Cane's • Miami, FL • 2h ago

Retail & Hospitality

Company Description Starting from $60,000 annually plus monthly training incentive of $750 Pay is based on location, experience, and qualifications etc. *Monthly incentives after training vary and are based on restaurant...

2h ago

Go to Apply

Assistant Manager (Indeed Event)

Panda Restaurant Group Inc • Kalispell, MT • 2h ago

Retail & Hospitality

Are You Craving A Career With An Industry Leader?Panda Express is searching for experienced leaders to join our store management team. Panda is 100% committed to the professional and personal development of our family of...

2h ago

Go to Apply

Subscribe to get new jobs sent directly to you
(no charge, cancel anytime)